DevOps Pipeline - Beginner's Guide
Below I will leave a step-by-step guide to running a DevOps pipeline, from code to deployment, both via video and in writing.
Requirements
Settings
1st Step: Make Vs Code connect to Github via an SSH token.
So to create an ssh token, you will run the code below in the terminal, replacing it with your email address used on Github.
ssh-keygen -t rsa -b 4096 -C "youremail@exemple.com"
After running the command, just press enter until the token is generated, as shown in the print:
Once this is done, it will save the SSH key on your computer, in the .ssh directory within your user. In our case we used the default, so the file will be id_rsa.pub. Then just open the file (you can do it with Notepad) and copy all its contents.
With the copied content, open the Github settings and go to SSH and GPG keys, and click on: New SSH Key.
Once done, just paste the copied code and give it a name. In the example we used vscode.
To test whether the connection using the token was successful, simply run the code below in the terminal.
ssh -T git@github.com
The system will say that it does not know the token and will ask if you still want to continue the connection. You can type "yes" and it will continue with the test. If successful, it will return the user's name with a success message, as shown in the image below.
2nd Step: Make a copy of the project to the local machine.
You will open the project that we will use as an example on github: https://github.com/brunnox/devops_pipeline
After that, you will create a Fork of the project, as shown in the print below.
After the fork, you will have a copy of the project in your own repository, where you can edit it and do whatever you want. So, to work with this project, we will clone this repository from Github to our local machine, so that we can edit it in VS Code.
To do this, on the project page, you must click on the arrow next to Code, then on SSH and finally copy the code that it shows, as shown in the image.
Now with this code copied, open a terminal and navigate to the directory where you want to save your project and pass the following code, replacing the content after git clone, with the content you copied.
git clone git@github.com:userexemple/devops_pipeline.git
After that, it will save a copy of your Github project to your local machine.
3rd Step: Create a project on Google Cloud, create users and grant permissions.
When you log into the Google Cloud Console, you click on Select a project, and then click on NEW PROJECT.
You can give the project any name, in the example here, I will use testpipeline. Once done, select the project and copy the Code (write it down somewhere, we will use it later).
Then search in the top bar: App Engine and select the first result.
We will use App Engine to run our application, select the server region that is closest to you, in my case I will use the South America server, then click to create a new service account.
You can give the account whatever name you want, the important thing is that it has these documents:
Then just go ahead and finish, then as soon as it generates the user you will go to the KEYS tab.
Click on add key and choose the option to Create a new key, as soon as a popup opens, you select the JSON option and click on create, it will download a file, leave it saved, which we will use later.
Go back to the App Engine screen, click the update button and select the service account you created and then click next.
Then search for APIs and Services in the top bar.
Access the Library tab.
Search for the first API: App Engine Admin API and enable it.
Search for the second API: Cloud Build API and enable it.
Explaining the Application:
Our application is very simple, a python application using flask.
In our application we have a requirements.txt file where we place the Python libraries that we are going to use in our application, we also have a file called test.py that will run a test as if it were a client accessing the application.
But the file that will make everything work is the pipeline.yml file that is inside the static directory. This file is responsible for running the entire pipeline in Github actions, and it is divided into 3 jobs.
- Lint: It calls up an Ubuntu machine, calls Actions, configures Python and installs dependencies, and runs indentation tests and searches for errors in the code.
- Tests: If the Lint job passes successfully, it calls an Ubuntu machine, calls Actions, configures and tests several versions of Python using an array, installs the dependencies, and runs the test.py file.
- Deploy: If the Tests job passes successfully, it calls an Ubuntu machine, calls Actions, makes connections to GCP and deploys the application to App Engine using Cloud Build.
In this file, in the connection part with GCP, you need to pass the Google Cloud project ID and also the generated Json key, however, as we are going to upload this code, the ideal is to use Github Actions secrets to mask this data, for the project ID we will use GCP_PROJECT_ID and for the Json key we will use GCP_SA_KEY.
Deploy
To deploy the application, there is very little left. The first adjustment is to pass the masked data of the project ID and the Json file to Github.
When accessing the project on Github, go to the settings screen, then access Secrets and Variables and click on New repository secret.
You will create 2 secrets, one for Json, which will be GCP_SA_KEY and in the Secret you will paste all the contents of the Json file.
And the second will be the GCP_PROJECT_ID where you will pass the project ID.
Now, to finish the deployment and get the application running, we need to make an adjustment. Github only understands that the application has a pipeline if the pipeline file is in a specific directory. You will create a directory in the root folder of your project with the following name ".github". Then, within this directory, you will create a new directory called "workflows" and then you will move the pipeline.yml file that is in the static directory to the workflows directory.
Once this is done, we can begin the process of deploying our application. To do this, we will run 3 commands in the terminal within our directory.
Add all files to the staging environment inside git.
git add .
Uploads changes from the staging environment to the master environment.
git commit -m "Add Workflow"
And finally, to upload our changes from the local repository to Github.
git push origin master
Now when accessing our Github repository again and going to the Actions tab, we will see that there is a workflow running.
Within the workflow we can see the 3 jobs mentioned above, and we can see that one job depends on the other to run.
By clicking on any of the jobs that are being executed, you can see the details.
After the deployment is complete, we just need to access App Engine again and we will have our application. To test the application, just click on the link that App Engine provides.
To finish here, we can see a print of the application working after having its deployment executed by Github Actions.
If this content was useful to you, share it with more people and follow me on social media so I can continue bringing more content to you.